Temporal Models and Smoothing
Data are often observed in time, and time dependence is often expected.
Note: We can use the same model to smooth covariate effects!
Smoothing of the time effect
Prediction
We can “predict” any unobserved data, does not have to be in the future
Time can be indexed over a
Discrete domain (e.g., years)
Continuous domain
Time can be indexed over a
Discrete domain (e.g., years)
Main models: RW1, RW2 and AR1
NB RW1 and RW2 are also used for smoothing
Continuous domain
Goal we want understand the pattern and predict into the future
Random walk models encourage the mean of the linear predictor to vary gradually over time.
They do this by assuming that, on average, the time effect at each point is the mean of the effect at the neighboring points.
Random Walk of order 1 (RW1) we take the two nearest neighbors
Random Walk of order 2 (RW2) we take the four nearest neighbors
Definition
\[ \begin{aligned} \pi(\mathbf{u} \mid \sigma^2) & \;\propto\; \exp\!\left( -\frac{1}{2\sigma^2} \sum_{t=1}^{T-1} (u_{t+1} - u_t)^2 \right)\\ & \;=\; \exp\!\left( -\frac{1}{2\sigma^2} \sum_{t \sim t'} ({u_t - u_{t'}})^2 \right) = \exp\!\left(-\tfrac{1}{2} \, \mathbf{u}^{\top} \mathbf{Q}\ \mathbf{u}\right) \end{aligned} \] where \(t \sim t'\) indicates \(t\) is a neighbor of \(t'\), and the precision is \(\mathbf{Q} = \mathbf{R}/\sigma^2\) with
\[ \mathbf{R} = \begin{bmatrix} 1 & -1 & & & & \\ -1 & 2 & -1 & & & \\ & -1 & 2 & -1 & & \\ & & \ddots & \ddots & \ddots & \\ & & & -1 & 2 & -1 \\ & & & & -1 & 1 \end{bmatrix} \]